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Description 

The present invention relates generally to appara- 
tus for applying lines of desired configuration onto a 
roadway or pavement surface and more particularly to 
apparatus for controlling application of paint or other line 
marking materials to accurately repeat previously 
formed lines on the roadway or pavement surface. 

Roadway or pavement marking lines (either centre 
lines or margin lines) inevitably wear out over time and 
the rehabilitation of these marking lines is a task that 
must be continually carried out by roadway mainte- 
nance organisations. The length of roadways in many 
countries including Australia result in a need for machin- 
ery that can accurately reform old line markings at rela- 
tively high speed so that costs can be minimised. There 
are in fact many difficulties involved in being able to ac- 
curately reform old making lines at high speed and some 
of these include difficulties with distinguishing the old 
line itself caused by extraneous road markings such as 
changes in road surface colouring, shadows, unusual 
colouring areas such as oil or paint splashes, and simply 
in some cases as a result of extreme wear of the old 
line. Moreover, changes in pattern configurations also 
cause difficulties for continuous high speed reforming of 
old lines. 

In prior patent literature there have been disclosed 
many proposals for mechanising reforming of old pave- 
ment lines. Some of these are disclosed in documents 
US-A-3046854, US-A-3101175, US-A-3286928 and 
US-A-3298352, all of which would have various prob- 
lems associated with being able to distinguish old line 
markings, to be able to accurately repeat those old line 
markings and more particularly, to handle changes in 
line patterns. In document US-A-5,054,959 there is dis- 
closed apparatus for distinguishing old line markings on 
a roadway surface and in response thereto, correctly po- 
sitioning paint applicator guns so that paint could be ap- 
plied correctly over the old line markings. This apparatus 
has been used successfully but certain difficulties have 
arisen in relation to recognition of old line markings in- 
cluding pattern changes in those markings, particularly 
when operating at high speed. 

The objective therefore of the present invention is 
to provide improvements in control apparatus for ma- 
chinery of the type generally disclosed in US-A- 
5,054,959 to improve recognition of old line markings 
and particularly changes in pattern formats of those line 
markings. 

Preferably it is an objective of the present invention 
to enable operational speeds to be improved by im- 
proved old line and line pattern recognition. 

According to the present invention there is provided 
control apparatus for controlling application of line mark- 
ing material to refurbish old line markings on a pavement 
surface, said control apparatus including line detection 
means adapted in use to transverse scan a predeter- 
mined width of said pavement surface substantially 



greater than a width of said old line pattern while being 
carried forwardly in the direction of said old line pattern, 
said control apparatus being characterised in that said 
line detection means includes means for comparing im- 
s ages derived from two image detector means, one 
mounted behind the other in the direction of travel, and 
including image classifier means taught through an ar- 
tificial neural network to recognise line patterns and to 
classify transitions in the old line marking, from one pat- 
10 tern to another, and said apparatus, in response thereto, 
being effective to control line marking material applica- 
tor means to repeat accurately said old line markings 
including transitions in the patterns thereof, with chang- 
es of pattern being made only if said detected line mark- 
's ing patterns are recognized by the control apparatus as 
taught by said neural network. 

Preferably, said control apparatus according to 
Claim 1 is further characterised in that a first one of said 
image detector means is mounted in use at a forward 
20 position whereby a first predetermined width of said 
pavement surface substantially greater than said old 
pattern width is transverse scanned, with information in- 
dicative of said old line pattern viewed by said first image 
detector means being retained over a first predeter- 
2S mined length of said pavement surface in the direction 
of said old line pattern, a said second one of said image 
detector means being mounted in use rearwardly of said 
first image detector means, said second image detector 
means being in use arranged to transverse scan a sec- 
30 ond predetermined width of said pavement surface sub- 
stantially greater than the width of said old line pattern, 
with information indicative to said old line pattern viewed 
by said second image detector means being retained 
over a second predetermined length of said pavement 
35 surface in the direction of said old line pattern, said first 
image detector means further recognising transition 
points in said old line pattern and providing a signal in- 
dicative of said transition points to said second image 
detector means, said second image detector means re- 
40 ceiving the signal indicative of transition points from said 
first image detector means and information from the 
transverse scan of said second image detector means 
and in response thereto, a control signal means is pro- 
vided to control on/off conditions of said line marking 
^5 material applicator means. 

According to a preferred feature of the present in- 
vention, said first image detector means produces a lat- 
eral shift control signal indicative of lateral shift of said 
old line pattern as sensed, with said lateral shift control 
50 signal being adapted to control lateral movement of line 
marking material applicator means to correctly deposit 
said line marking material over lines forming said old line 
pattern. 

Preferably the first predetermined length of 
55 scanned pavement surface is substantially greater than 
said second predetermined length. Conveniently said 
first predetermined length is about five metres in length. 
The first predetermined width and the second predeter- 
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mined widths may be equal or they may be different 
however, they are preferably between 400 mm and 1 .2 
metres, conveniently about 500 mm. 

The present invention of course includes within its 
scope apparatus for applying line marking materials to 
old line patterns including control apparatus as afore- 
said carried by a vehicle or the like with line marking 
material applicator means located rearwardly of the sec- 
ond line detection means. 

The applicator means may be constructed as de- 
scribed in US-A-5,054,959 or any other suitable means 
could be used. The line marking material might be liquid 
paint including solvent based paints or water based 
paints, thermally or cold applied plastic strip in one part 
or two component mix, or any other suitable material for 
refurbishing old lines. 

A preferred embodiment will hereinafter be de- 
scribed with reference to the accompanying schematic 
diagrams, in which: 

Figure 1 represents an information flow diagram in 
a preferred form of control apparatus according to 
the present invention; 

Figure 2 represents schematically operation of a 
survey system line detection apparatus forming part 
of the system illustrated in Figure 1 ; 
Figure 3 represents schematically operation of a 
control system line detection apparatus forming a 
further part of the system illustrated in Figure 1 , and 
Figure 4 represents schematically operation of a 
mechanical system controlled by the control system 
for correctly actuating the line marking material ap- 
plicators. 

Apparatus according to the present invention may 
be mounted on or from a suitable vehicle such as a truck 
or the like capable of carrying quantities of line marking 
materials to refurbish lines on a desired length of road- 
way or pavement surface. The form of mounting may 
include a forwardly mounted and extending boom such 
as in US-A-5,054,959, but other carrying arrangements 
might also be used. The control apparatus conveniently 
includes a forwardly located survey system, a mid locat- 
ed control system (relative to vehicle travel/old line pat- 
tern direction), both of which are used to control a rear- 
wardly located mechanical system for positioning and 
controlling application of the line marking material. Pref- 
erably, visual left and right trajectory guides are provided 
capable of being viewed by the driver of the vehicle so 
that the driver may conveniently locate the vehicle dur- 
ing high speed operation so that the old line pattern on 
the pavement surface passes within a predetermined 
operational width defined by the survey system. 

The survey system includes a forwardly (in use) 
mounted CCD camera which is generally sensitive to 
the overall pattern of the old line markings in the pave- 
ment surface. The CCD camera of the survey system is 
driven from the carrying vehicles transmission (or is oth- 



erwise responsive to speed of operation of the carrying 
vehicle), and acquires a transverse line scan repeatedly 
over short discrete lengths of road surface, for example, 
every five mm. The optics and mounting geometry of the 
5 CCD camera of the survey system map repeated trans- 
verse line scans to approximately half a metre of road 
surface width. The survey system further includes an im- 
age manager that buffers and aggregates the CCD cam- 
era pixels into an array, 13 across (transverse) by 5 in 

10 the direction of vehicle travel. Each of these pixels may 
be about 40 mm wide (giving a transverse scan of 520 
mm total) and about 1 metre long (giving a longitudinal 
scan of about 5 metres total). 

The survey system also includes a lateral compen- 

t5 sator which compresses a copy of the aforesaid pixel 
array into an array of 13 across and one (1) longitudi- 
nally. This is achieved by averaging along each column 
of the five by one metre pixels. The lateral compensator 
has conveniently been taught, through a neural network, 

20 to recognise lateral drift of the road surface image and 
to compensate for such lateral drift. The signal from the 
lateral compensator is applied both to the mechanical 
system and to the image classifier of the survey system 
to rotate left/right so that they are laterally central to their 

2$ respective pixel frames. The image classifier has been 
taught, through a neural network, to classify transitions 
from one paint pattern to the next in an old pattern on 
the roadway surface. Transitions are defined by a state- 
transition diagram. Any transition detected by the survey 

30 system is signalled ahead to the control system so that 
it knows what kind of transition to look for. 

As indicated above, the lateral compensation signal 
is also passed to the mechanical system which moves 
the line marking material applicator means left or right 

35 to position the line marking material applicator means 
correctly over the old line (or lines). 

At any time, the current track is indicated by a set- 
ting manually input by the driver (or operator). This de- 
termines which line material applicator (paint supply gun 

40 in the case of liquid paint being applied) is in operation 
for a single paint line (either continuous or intermittent). 
If a wrong track is detected by the survey system, it is- 
sues an override signal which switches off the applica- 
tors temporarily and connects the tracking by reposition- 

45 jng the carriage carrying the applicators (see for exam- 
ple US-A-5,054,959). Furthermore, the lateral compen- 
sator can recognise paint images at the extremities of 
the CCD camera field of view. It will then issue an emer- 
gency shut down signal to the applicators (paint guns). 

so Referring now to Figure 4, the control system com- 
prises a CCD camera, driven from the vehicle transmis- 
sion (or some other vehicle speed controlled input) and 
acquires a transverse line scan approximately every 5 
mm. The optics and mounting geometry map a trans- 

55 verse line scan to approximately half a metre of road 
width. The control system includes an image manager 
that buffers and aggregates the CCD pixels into an array 
13 across by 10 along the direction of travel. Each pixel 
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in the array may be approximately 40 mm wide giving a 
total width in the array of 520 mm and 30 mm long giving 
an approximate length of the array of 300 mm. The in- 
coming lateral compensation signal causes images in 
the Transition Detector to rotate left/right so that they 
are laterally central to their pixel frame. 

The Transition Detector has been taught, through a 
Neural Network, to recognise the exact point at which a 
transition occurs between one line pattern and the next. 
The Transition Detector uses not only the buffered im- 
age from the CCD camera of the control system, but also 
the incoming pattern classification from the survey sys- 
tem, to determine what kind of signal to send to the line 
marking material applicators. The applicator signals 
may be: 



"P" = pulse - 



on for 3m and off for 8.5 m which is 
the normal line/spacing configura- 
tion for intermittent road lines in 
Australia 



n O" = 
"C" = 



open 
close 



Figure 4 illustrates schematically the control system 
for the mechanical system which may be constructed as 
in US-A-5,054,959. In the following discussion refer- 
ence is made to paint guns, however, other forms of ap- 
plicators could be used depending on the nature of the 
line marking material used. The paint gun actuators are 
signalled open/closed by the gun control module. It re- 
ceives signals O, C or P from the control system. "P - (or 
pulse) results in a timed open-close sequence from the 
paint gun control module. The paint guns can be shut 
down (until manually reset by the driver/operator) with 
a gun shut down signal from the survey system. This is 
a fail-safe mechanism to avoid out of range paint appli- 
cation. When a track change over-ride signal is received 
from the survey system, the paint guns momentarily 
shut off to allow the gun carriage to take up its correct 
position. Lateral control signals cause the gun carriage 
to move left or right for lateral compensation of the ve- 
hicles trajectory. Both the gun control module and the 
carriage control module have control loops to effect 
proper gun actuator movement and carriage position. 



Claims 

1 . Control apparatus for controlling application of line 
marking material to refurbish old line markings on a 
pavement surface, said control apparatus including 
line detection means adapted in use to transverse 
scan a predetermined width of said pavement sur- 
face substantially greater than a width of said old 
line pattern while being carried forwardly in the di- 
rection of said old line pattern, 

said control apparatus being characterised in 
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that said line detection means includes means for 
comparing images derived from two image detector 
means, one mounted behind the other in the direc- 
tion of travel, and including image classifier means 
taught through an artificial neural network to recog- 
nise line patterns and to classify transitions in the 
old line marking, from one pattern to another, and 
said apparatus, in response thereto, being effective 
to control line marking material applicator means to 
repeat accurately said old line markings including 
transitions in the patterns thereof, with changes of 
pattern being made only if said detected line mark- 
ing patterns are recognized by the control appara- 
tus as taught by said neural network. 

Control apparatus according to Claim 1, further 
characterised in that a first one of said image de- 
tector means is mounted in use at a forward position 
whereby a first predetermined width of said pave- 
ment surface substantially greater than said old pat- 
tern width is transverse scanned, 

with information indicative of said old line pat- 
tern viewed by said first image detector means 
being retained over a first predetermined length 
of said pavement surface in the direction of said 
old line pattern, 

a said second one of said image detector 
means being mounted in use rearwardly of said 
first image detector means, said second image 
detector means being in use arranged to trans- 
verse scan a second predetermined width of 
said pavement surface substantially greater 
than the width of said old line pattern, 
with information indicative to said old line pat- 
tern viewed by said second image detector 
means being retained over a second predeter- 
mined length of said pavement surface in the 
direction of said old line pattern, 
said first image detector means further recog- 
nising transition points in said old line pattern 
and providing a signal indicative of said transi- 
tion points to said second image detector 
means, said second image detector means re- 
ceiving the signal indicative of transition points 
from said first image detector means and infor- 
mation from the transverse scan of said second 
image detector means and in response thereto, 
a control signal means is provided to control on/ 
off conditions of said line marking material ap- 
plicator means. 

Control apparatus according to Claim 2, character- 
ised in that said first image detector means further 
produces a lateral shift control signal indicative of a 
lateral shift of said old line pattern as sensed with 
said lateral shift control signal being adapted tocon- 
trol lateral movement of line marking material appli- 
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cator means to correctly deposit said line marking 
material over lines forming said old line pattern. 

4. Control apparatus according to Claim 2 or Claim 3, 
wherein said first predetermined length of scanned 
pavement surface is substantially greater than said 
second predetermined length. 

5. Control apparatus according to Claim 4, further 
characterised in that said first predetermined length 
is about five metres. 

6. Control apparatus according to Claim 4 or Claim 5, 
further characterised in that said first and second 
predetermined widths are between 400 mm and 
1200 mm. 

7. Apparatus for applying line marking material to re- 
furbish an old line pattern on a pavement surface, 
said apparatus including a control apparatus ac- 
cording to any one of the preceding claims. 



Patentanspruche 

1. Steuervorrichtung zum Steuern der Aufbringung 
von Linienmarkierungsmaterial fur die Wiederher- 
stellung von alten Linienmarkierungen auf einer 
Fahrbahnflache, wobei die Steuervorrichtung eine 
Linienerfassungseinrichtung enthalt, die bei Betrieb 
eine vorbestimmte Breite der Fahrbahnflache in 
Querrichtung abtasten kann, die wesentlich grofter 
ist als die Breite des alten Linienmusters, wahrend 
sie in Richtung des alten Linienmusters vorwarts 
gefuhrt wird, 

wobei die Steuereinrichtung dadurch gekenn- 
zeichnet ist, daft die Linienerfassungseinrichtung 
eine Einrichtung zum Vergleichen von Bildern ent- 
halt, die von zwei Bitderfassungseinrichtungen er- 
halten werden, die hintereinander in Bewegungs- 
richtung angebracht sind und Bildklassierungsein- 
richtungen enthalten, welche durch ein neuronales 
Netzwerk belehrt werden, urn Linienmuster zu er- 
kennen und Ubergange in der alten Linienmarkie- 
rung von einem Muster zu einem anderen zu klas- 
sieren, und daft die Vorrichtung in Abhangigkeit 
hiervon tatig wird, die Aufbringeinrichtung fur das 
Linienmarkierungsmaterial zu steuern, urn die alten 
Linienmarkierungen einschlieftlich der Ubergange 
in deren Muster genau zu wiederholen, wobei Mu- 
steranderungennurdurchgefuhrt werden, wenndie 
erfaBten Linienmarkierungsmuster von der Steuer- 
vorrichtung als durch das neuronale Netzwerk ge- 
lehrt erkannt werden. 

2. Steuereinrichtung nach Anspruch 1, dadurch ge- 
kennzeichnet, daft eine erste der Bitderfassungs- 
einrichtungen bei Betrieb in einer Vorwartsanord- 



nung angebracht ist, wodurch eine erste vorbe- 
stimmte Breite der Fahrbahnflache, die wesentlich 
grafter ist als die Breite des alten Musters, in Quer- 
richtung abgetastet wird, 

5 

daft Information, die das von der ersten Bilder- 
fassungseinrichtung betrachtete alte Linienmu- 
ster angibt, uber eine erste vorbestimmte Lan- 
ge der Fahrbahnflache in Richtung des alten Li- 
io nienmusters beibehatten wird, 

daft eine zweite der Bilderfassungseinrichtun- 
gen bei Betrieb von der ersten Bilderfassungs- 
einrichtung ruckwarts angebracht ist, wobei die 
zweite Bilderfassungseinrichtung bei Betrieb 
so angeordnet und eingerichtet ist, daft sie eine 
zweite vorbestimmte Breite der Fahrbahnfla- 
che in Querrichtung abtastet, die wesentlich 
groBer ist als die Breite des alten Linienmu- 
sters, 

^0 daft Information, die das von der zweiten Bil- 

derfassungseinrichtung betrachtete alte Lini- 
enmuster angibt, uber eine zweite vorbestimm- 
te Lange der Fahrbahnflache in Richtung des 
alten Linienmusters beibehatten wird, 

25 und daft die erste Bilderfassungseinrichtung 

ferner Ubergangspunkte im alten Linienmuster 
erkennt und ein die Ubergangspunkte ange- 
bendes Signal auf die zweite Bilderfassungs- 
einrichtung gibt, wobei die zweite Bilderfas- 

30 sungseinrichtung das die Ubergangspunkte 

angebende Signal von der ersten Bilderfas- 
sungseinrichtung sowie Information von der 
Querabtastung der zweiten Bilderfassungsein- 
richtung empfangt und in Abhangigkeit davon 

35 eine Steuersignaleinrichtung den Einschalt/ 

Ausschaltzustand der Aufbringeinrichtung fur 
das Linienmarkierungsmaterial steuert. 

3. Steuervorrichtung nach Anspruch 2, dadurch ge- 
40 kennzeichnet, daft die erste Bilderfassungseinrich- 
tung ein Seitenverschiebungs-Steuersignal er- 
zeugt, das eine Seitenverschiebung des alten Lini- 
enmusters gemaft der Abtastung anzeigt, wobei 
das Seitenverschiebungs-Steuersignal zur Steue- 

45 rung einer seitlichen Verschiebung der Aufbringein- 
richtung fur das Linienmarkierungsmaterial geeig- 
net ist, urn das Linienmarkierungsmaterial genau 
uber das alte Linienmuster bildenden Linien aufzu- 
bringen. 

so 

4. Steuervorrichtung nach Anspruch 2 Oder 3, bei wel- 
cher die vorbestimmte Lange der abgetasteten 
Fahrbahnflache wesentlich groBer ist als die zweite 
vorbestimmte Lange. 

55 

5. Steuervorrichtung nach Anspruch 4, dadurch ge- 
kennzeichnet, daft die erste vorbestimmte Lange 
etwa 5 m betragt. 
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Steuervorrichtung nach Anspruch 4 Oder 5, da- 
durch gekennzeichnet, daG die erste und zweite 
vorbestimmte Breite zwischen 400 mm und 1200 
mm liegen. 

Vorrichtung zum Aufbringen von Linienmarkie- 
rungsmaterial zur Wiederherstellung eines atten Li- 
nienmusters auf einer Fahrbahnflache, wobei die 
Vorrichtung eine Steuervorrichtung nach einem der 
vorangehenden Anspruche enthalt. 



Revendicatlons 

1 . Dispositif de commande pour contr6ler I'application 
de materiau de marquage de ligne afin de restaurer 
d'anciens marquages de lignes sur une surface de 
chauss6e, ledit dispositif de commande compre- 
nant des moyens de detection de ligne adapt6s en 
service pour balayer transversalement une largeur 
pr6determin6e de ladite surface de chaussde sen- 
siblement sup6rieure k une largeur dudit ancien 
motif de ligne tout en 6tant d6plac6s vers I'avant 
dans la direction dudit ancien motif de ligne, 

ledit dispositif de commande 6tant caracterise 
en ce que lesdits moyens de detection de ligne 
comprennent des moyens pour comparer des ima- 
ges obtenues de deux moyens detecteurs d'image, 
Tun monte derri&re I'autre dans la direction de de- 
placement, et comprenant des moyens de classifi- 
cation d'image pilotes par un r6seau neuronal afin 
de reconnattre des motifs en ligne et de classer des 
transitions dans Pancien marquage de ligne, d'un 
motif k I'autre, et ledit dispositif, en reaction k cela, 
agissant pour contrdler des moyens applicateurs de 
materiau de marquage de ligne afin de reproduire 
avec precision lesdits anciens marquages de ligne, 
y compris les transitions dans leurs motifs, des mo- 
difications de motif n'6tant apportees que si lesdits 
motifs de marquage de ligne d6tect6s sont recon- 
nus par le dispositif de commande tel qu'il est pilots 
par ledit r6seau neuronal. 



2. Dispositif de commande seion la revendication 1, 
caracterise en ce qu'un premier desdits moyens d6- 
tecteurs d'image est monte en service dans une po- 
sition avant, de sorte qu'une premiere largeur pre- 
determine de ladite surface de chaussee sensible- 
ment sup6rieure k la largeur dudit ancien motif est 
balayee transversalement, 

des informations representatives dudit ancien 
motif de ligne vu par lesdits premiers moyens 
detecteurs d'image etant relev6es sur une pre- 
miere longueur pr6determin6e de ladite surfa- 
ce de chaussee dans la direction dudit ancien 
motif de ligne, 

ledit deuxieme desdits moyens detecteurs 
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d'image 6tant monte en service k Parri&re des- 
dits premiers moyens detecteurs d'image, les- 
dits deuxiemes moyens detecteurs d'image 
etant agenc6s en service pour balayer trans- 
versalement une deuxieme largeur pr6determi- 
n6e de ladite surface de chauss6e sensible- 
ment sup6rieure k la largeur dudit ancien motif 
de ligne, 

des informations representatives dudit ancien 
motif de ligne vu par lesdits deuxi6mes moyens 
detecteurs d'image etant relev6es sur une 
deuxieme longueur pr6d6termin6e de ladite 
surface de chaussee dans la direction dudit an- 
cien motif de ligne, 

lesdits premiers moyens detecteurs d'image 
reconnaissant de plus des points de transition 
dans ledit ancien motif de ligne et deiivrant un 
signal reprdsentatif desdits points de transition 
auxdits deuxiemes moyens detecteurs d'ima- 
ge, lesdits deuxiemes moyens detecteurs 
d'image recevant le signal repr6sentatif de 
points de transition depuis lesdits premiers 
moyens detecteurs d'image et des informations 
par le balayage transversal desdits deuxiemes 
moyens detecteurs d'image et en reaction k ce- 
la, des moyens de signal de commande sont 
pr6vus pour contrdler l'6tat marche/arrSt des- 
dits moyens applicateurs de materiau de mar- 
quage de ligne. 

Dispositif de commande selon la revendication 2, 
caracterise en ce que lesdits premiers moyens de- 
tecteurs d'image produisent de plus un signal de 
commande de deplacement lateral repr6sentatif 
d'un deplacement lateral dudit ancien motif de ligne 
detects, ledit signal de contr6le de d6p!acement la- 
teral etant adapte pour contrdler le mouvement la- 
teral des moyens applicateurs de mat6riau de mar- 
quage de ligne afin de deposer correctement ledit 
materiau de marquage de ligne par-dessus des li- 
gnes formant ledit ancien motif de ligne. 

4. Dispositif de commande selon la revendication 2 ou 
3, dans lequel ladite premiere longueur pr6d6termi- 
n6e de surface de chaussee balay6e est sensible- 
ment sup6rieure k ladite deuxieme longueur pr6d6- 
terminee. 

5. Dispositif de commande selon la revendication 4, 
caracterise en ce que ladite premiere longueur pre- 
d6terminee est d'environ cinq metres. 

6. Dispositif de commande selon la revendication 4 ou 
5, caracterise en ce que lesdites premiere et 
deuxieme largeurs pr6d6termin6es sont comprise 
entre 400 mm et 1 200 mm. 

7. Dispositif pour appliquer du materiau de marquage 



11 



EP 0 632 858 B1 



de ligne afin de restaurer un ancien motif de ligne 
sur une surface de chaussde, ledit dispositif com- 
prenant un dispositif de commando selon Tune 
quetconque des revendications pr6c6dentes. 
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